feat(FieldFormatters): add visual editor #5075

maxpatiiuk · 2024-07-07T00:30:23Z

Fixing an issue from 2012 😊

Fixes #23

Checklist

Self-review the PR after opening it to make sure the changes look good
and self-explanatory (or properly documented)
Add automated tests
Add relevant issue to release milestone

Testing instructions

Generally visual editor for Field Formatters should act and look very similar to the one for Record formatters since it shares a lot of code with it.

To test:

Verify that each field formatter type (constant, any character, ...) can be set in the UI, without user-experience issues, and after saving, can be read back correctly
Verify that a formatter created/edited using a visual editor works in Specify 7 and Specify 6 (in forms and query results)
- For reference, here is how to configure a field formatter in Specify 6
P.S: don't try to enter an emoji character like on the screenshot above - Specify 7 back-end does not support emojis in app resources 😢

Documentation on field formatters

maxpatiiuk · 2024-07-07T00:32:39Z

specifyweb/frontend/js_src/lib/components/FieldFormatters/FieldFormatter.tsx

+            if (mapping !== undefined && mapping?.length > 1)
+              softError('Expected mapping length to be no more than 1');
+            const field = mapping?.[0];
+            if (field?.isRelationship === true) {
+              softError(
+                'Did not expect relationship field in field formatter mapping'
+              );


these cases should not happen - the UI doesn't allow them. but handling them here just in case (and to satisfy TypeScript)

specifyweb/frontend/js_src/lib/components/SchemaConfig/Format.tsx

grantfitzsimmons · 2024-07-08T13:12:44Z

❤️

grantfitzsimmons

When creating a field format like GiftNumber using the regular expression /^(KUI|KUBI|NHM)$/, I am unable to set it up correctly.

See that I am supposed to put the 'hint' on the left and 'pattern' on the right:

This works as expected when using a regex validator, but when setting it as the pattern it is not recognized properly.

	<format system="true" name="GiftNumber" class="edu.ku.brc.specify.datamodel.Gift" fieldname="giftNumber" title="" default="true">
		<autonumber>edu.ku.brc.af.core.db.AutoNumberGeneric</autonumber>
		<field type="regex" size="4" value="/^(KUI|KUBI|NHM)$/" byyear="true" pattern="KUBI"/>
	</format>

After modifying other sections of the format, those changes are not made immediately visible for the 'Example Field'. Instead, it uses the previous field format before the edits have been made.

See that it shows "Required Format: KUBI" in the tooltip:

(This tooltip is most often obscured by the browser's "match the requested formatter" error)
https://github.com/specify/specify7/assets/37256050/d34bcb16-85d5-43fc-8c68-56bef2e36cc6

After making it simply numeric:

What does the 'Auto Numbering' do that the 'Auto-number' checkbox does not?

maxpatiiuk · 2024-07-13T15:07:28Z

When creating a field format like GiftNumber using the regular expression /^(KUI|KUBI|NHM)$/, I am unable to set it up correctly.

Looks like the expected regex between Sp6 and Sp7 differed.
In Specify 6, the regex is run on each field part separately, so /^ and $/ are necessary
In Specify 7, all regexes are joined together into on regex, thus, having /^ or $/ will break things if there is more than one part in your formatter.

Fixed:

When reading from .xml, both Specify 7 front-end and back-end will trim any /^ or $/ from the regex pattern to not break the pattern when multiple parts are joined together. Similarly, outer ( and ) are removed if present
When front-end saves the updated .xml, the syncers will make sure to add /^ and $/, even if not originally specified by the user. Also, if the pattern includes |, leading ( and trailing ) will be added

In practice, that means:

When using the visual field format editor, it doesn't matter anymore if you put or not put /^ and $/ in a regex field as the value will be normalized automatically
Backwards compatibility with Specify 6

maxpatiiuk · 2024-07-13T15:11:49Z

See that I am supposed to put the 'hint' on the left and 'pattern' on the right:

yeah, it's not ideal that 'hint' comes before the 'pattern' as the opposite order seems more intuitive.

The reason I did it this way is because of the column heading.
For all other part types, the 'hint' column is used both for declaring what the format should be, and for proving the validation message
For regex parts, since regex is not "user friendly" enough to put into a regex message, the 'hint' field is used only for validation messages, and the regex pattern itself is a separate field (which also validates that the regex pattern has correct syntax)

Open to ideas on how to make this more intuitive

maxpatiiuk · 2024-07-13T15:28:55Z

After modifying other sections of the format, those changes are not made immediately visible for the 'Example Field'. Instead, it uses the previous field format before the edits have been made.

Fixed

maxpatiiuk · 2024-07-13T15:33:42Z

What does the 'Auto Numbering' do that the 'Auto-number' checkbox does not?

Ups, the "Auto Numbering" checkbox at the top is not supposed to be exposed in the UI - removed
(it corresponds to whether .xml had a tag like <autonumber>edu.ku.brc.specify.dbsupport.CollectionAutoNumber</autonumber>, which is an sp6-only construct - sp7 cares about it only in so far as creating sp6-compatiable field formatters)

Fixes #5075 (review)

grantfitzsimmons · 2024-07-19T17:00:20Z

@maxpatiiuk Can you resolve the merge conflicts?

lexiclevenger

Testing instructions

Generally visual editor for Field Formatters should act and look very similar to the one for Record formatters since it shares a lot of code with it.

To test:

Verify that each field formatter type (constant, any character, ...) can be set in the UI, without user-experience issues, and after saving, can be read back correctly
Verify that a formatter created/edited using a visual editor works in Specify 7 and Specify 6 (in forms and query results)
- For reference, here is how to configure a field formatter in Specify 6
P.S: don't try to enter an emoji character like on the screenshot above - Specify 7 back-end does not support emojis in app resources 😢

Super exciting! I have a few recommendations:

If no default formatter is selected, the message says "record formatter. I would change this to "field formatter"

The default for the "size" field for most field types is 0, and this can be saved as long as there is a value in the "hint" field. The value will be shown in the formatter, but it will make the format invalid if that value is entered in the field. In the Specify 6 field formatter, the "size" value cannot go below 1; I think adding this in 7 would be a good solution.

The user can input and save any value at any length in the "hint" field (see screenshot above) for the numeric field type, but this is misleading because "#" is the only character that actually works, and it will only appear as many times as the value in the "size" field. Maybe this field could be read-only since it's not really functional.

I'd like to hear what others have to say about 2 and 3; they don't conflict with the xml editor, and the "Example field" displays their consequences correctly, but I think these solutions would make the visual editor more user-friendly.

maxpatiiuk · 2024-07-21T18:46:32Z

If no default formatter is selected, the message says "record formatter. I would change this to "field formatter"

Good point. To have the same message be applicable both in the field formatters editor and record formatters editor, I edited it to say:

"Please designate one of the formatters as default"

The default for the "size" field for most field types is 0, and this can be saved as long as there is a value in the "hint" field. The value will be shown in the formatter, but it will make the format invalid if that value is entered in the field. In the Specify 6 field formatter, the "size" value cannot go below 1; I think adding this in 7 would be a good solution.

Made "size" value not go below 1

The user can input and save any value at any length in the "hint" field (see screenshot above) for the numeric field type, but this is misleading because "#" is the only character that actually works, and it will only appear as many times as the value in the "size" field. Maybe this field could be read-only since it's not really functional.

Fixed. "hint" is now readonly for number fields

Fixes #5075 (review)

lexiclevenger

Looks good! Another thing I noticed is that clicking on the "edit" button next to a format in schema config often takes you to the wrong one.

Schema.Config_.Gift._.Specify.7.-.Google.Chrome.2024-07-23.11-30-30.mp4

https://fwri1924-field-editor.test.specifysystems.org/specify/schema-config/en/Gift/

- Field is ambiguous name - Field suggests that you can have fields inside of a formatter - you can't - you can only have text parts inside a field formatter

Triggered by 06b3c1a on branch refs/heads/field-editor

Fixes #5075 (review)

Triggered by b4736f8 on branch refs/heads/field-editor

Fixes #5075 (review)

Fixes #5075 (review) I should have known better than to use `useEffect` for this. Moved the logic to a more appropriate lace.

Fixes #5075 (review)

…y back-end

maxpatiiuk · 2025-01-27T00:57:07Z

If you shrink the dialog the preview background shrinks but if the value is long enough it extends past the background

Fixed by force-wrapping long values.

It is not a huge deal especially if it is hard to fix but if possible I would like the carrying over to be fixed

Could you clarify what you mean by fixed?
Do you want carrying over to be completely disabled? It is there as feature - if you say it is an anti feature, we can remove it
If by fixed you mean (looking at your example video) the size should revert back to 8 after being truncated to 4 be, the that is possible but a bit tricky.

Triggered by c8acc26 on branch refs/heads/field-editor

maxpatiiuk added this to the 7.9.7 milestone Jul 7, 2024

maxpatiiuk self-assigned this Jul 7, 2024

maxpatiiuk commented Jul 7, 2024

View reviewed changes

maxpatiiuk requested review from grantfitzsimmons, acwhite211 and a team July 7, 2024 00:56

maxpatiiuk removed this from the 7.9.7 milestone Jul 7, 2024

realVinayak reviewed Jul 7, 2024

View reviewed changes

specifyweb/frontend/js_src/lib/components/SchemaConfig/Format.tsx Outdated Show resolved Hide resolved

realVinayak requested a review from a team July 7, 2024 02:28

grantfitzsimmons requested changes Jul 8, 2024

View reviewed changes

maxpatiiuk requested review from a team and removed request for acwhite211 July 13, 2024 15:45

maxpatiiuk added a commit that referenced this pull request Jul 13, 2024

fix(FieldFormatters): normalize ^ and $ in regex

d8ef2a5

Fixes #5075 (review)

maxpatiiuk force-pushed the field-editor branch from f5b2b65 to b4736f8 Compare July 13, 2024 15:46

maxpatiiuk requested review from grantfitzsimmons and a team July 13, 2024 15:46

lexiclevenger requested changes Jul 19, 2024

View reviewed changes

lexiclevenger requested a review from a team July 19, 2024 19:42

maxpatiiuk added a commit that referenced this pull request Jul 21, 2024

fix(FieldFormatters): normalize ^ and $ in regex

9b664c9

Fixes #5075 (review)

maxpatiiuk added a commit that referenced this pull request Jul 21, 2024

fix(FieldFormatters): resolve minor UX issues

22ced94

Fixes #5075 (review)

maxpatiiuk force-pushed the field-editor branch from 91bf354 to 22ced94 Compare July 21, 2024 18:47

maxpatiiuk requested a review from lexiclevenger July 21, 2024 18:47

lexiclevenger requested changes Jul 23, 2024

View reviewed changes

maxpatiiuk added 26 commits January 26, 2025 16:42

feat(FieldFormatter): begin implementing visual editor

0fd3c95

refactor(FieldFormatters): cleanup placeholder & regex semantics

03acac0

refactor(FieldFormatter): rename "Field" to "Part"

4e0cec6

- Field is ambiguous name - Field suggests that you can have fields inside of a formatter - you can't - you can only have text parts inside a field formatter

feat(FieldFormatters): complete visual editor

ac18dc4

feat(FieldFormatters): improve visual editor UX

516a519

tests(FieldFormatters): update failing tests

6850bab

Lint code with ESLint and Prettier

4ae2f21

Triggered by 06b3c1a on branch refs/heads/field-editor

refactor(FieldFormatters): do a self code review

0b5cba0

feat(SchemaConfig): add link to FieldFormatter VisualEditor

c5eff48

fix(FieldFormatters): normalize ^ and $ in regex

0587e45

Fixes #5075 (review)

fix(FieldFormatters): correctly resolve formatter in editor preview

61ef5fd

fix(FieldFormatters): don't expose internal autoNumbering state in UI

1bdece9

Lint code with ESLint and Prettier

a118d30

Triggered by b4736f8 on branch refs/heads/field-editor

fix(FieldFormatters): resolve minor UX issues

7978624

Fixes #5075 (review)

fix(FieldFormatters): fix schema config indexes being off

1cfe2d8

Fixes #5075 (review)

fix(FieldFormatters): don't trigger "Save" without user changes

f7a6b3e

Fixes #5075 (review) I should have known better than to use `useEffect` for this. Moved the logic to a more appropriate lace.

docs(localization): fix __init__.py file name typo

7ac3ded

refactor(FieldFormatters): deduplicate formatter normalization logic

341915d

refactor(FieldFormatters): remove duplidate regex normalization logic

93eefca

fix(FieldFormatters): prevent CustomElementSelect not closing on click

d2dcc58

fix(FieldFormatters): set max field size at 9999

c4318aa

revert(localization): delete unused string

52365e1

fix(FieldFormatter): resolve reported issues

b7d3cb0

Fixes #5075 (review)

fix(FieldFormatters): clear stale regex validation error

3d4b511

fix(FieldFormatters): disable auto-numering in UI where not allowed b…

b95c145

…y back-end

fix(FieldFormatters): force-wrap long preview lines

c8acc26

maxpatiiuk force-pushed the field-editor branch from 6567197 to c8acc26 Compare January 27, 2025 00:57

maxpatiiuk requested a review from emenslin January 27, 2025 00:57

Lint code with ESLint and Prettier

37ca347

Triggered by c8acc26 on branch refs/heads/field-editor

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(FieldFormatters): add visual editor #5075

feat(FieldFormatters): add visual editor #5075

maxpatiiuk commented Jul 7, 2024 •

edited

Loading

maxpatiiuk Jul 7, 2024

grantfitzsimmons commented Jul 8, 2024

grantfitzsimmons left a comment

maxpatiiuk commented Jul 13, 2024 •

edited

Loading

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

grantfitzsimmons commented Jul 19, 2024

lexiclevenger left a comment

maxpatiiuk commented Jul 21, 2024 •

edited

Loading

lexiclevenger left a comment

maxpatiiuk commented Jan 27, 2025

feat(FieldFormatters): add visual editor #5075

Are you sure you want to change the base?

feat(FieldFormatters): add visual editor #5075

Conversation

maxpatiiuk commented Jul 7, 2024 • edited Loading

Checklist

Testing instructions

maxpatiiuk Jul 7, 2024

Choose a reason for hiding this comment

grantfitzsimmons commented Jul 8, 2024

grantfitzsimmons left a comment

Choose a reason for hiding this comment

maxpatiiuk commented Jul 13, 2024 • edited Loading

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

maxpatiiuk commented Jul 13, 2024

grantfitzsimmons commented Jul 19, 2024

lexiclevenger left a comment

Choose a reason for hiding this comment

Testing instructions

maxpatiiuk commented Jul 21, 2024 • edited Loading

lexiclevenger left a comment

Choose a reason for hiding this comment

maxpatiiuk commented Jan 27, 2025

maxpatiiuk commented Jul 7, 2024 •

edited

Loading

maxpatiiuk commented Jul 13, 2024 •

edited

Loading

maxpatiiuk commented Jul 21, 2024 •

edited

Loading